Modeling of spoken dialogue with and without visual information

Auditory information is the major factor in the human communication but in practical conversations, visual information such as gesture, facial expression, and head movement clearly makes it much smoother and more natural. Most researches related to analysis of spoken dialogue are based on only auditory information. We are trying to clarify how human utilize the knowledge of spoken dialogue management by dealing with more natural communication that includes visual information. Above all, by using visual information we can deal with listener's attitude against the speaker that cannot be done by using only auditory information.